1. Field of the Invention
The present invention relates to a human interactive type display system comprising a display means having a display panel on which an image is displayed, a marking means for pointing a point on said display panel, and a controlling means for detecting coordinates of the point on the display panel pointed by said marking means and controlling the image displayed on the display panel in accordance with the detected coordinates of the point pointed by the marking means.
2. Related Art Statement
In universities, colleges and laboratories, there conventional blackboard and whiteboards have been used as a medium for communicating information between a teacher and students or a presenter and attendants or participants. Typically in a university room, teachers use a blackboard or whiteboard for lectures. However, the blackboard or whiteboard could not satisfactorily communicate information between the teacher and students. In a modern computer room, each student uses his own personal computers. Therefore, the possibility of misreading and miscopying information can be reduced. However, there is a concern that some students would focus their attention on their personal computers and would not pay attention to a teacher""s explanations.
In order to solve the above mentioned problem, there have been proposed a human interactive type display system comprising a display panel displaying an image thereon, a pen-like member pointing or marking a point on the image displayed on the display panel, a means for detecting a contact condition of the pen-like member to the display panel and coordinates of a contact point, and a means for controlling the image displayed on the display panel in accordance with the detected contact condition and coordinates. Such a human interactive type display system is sometimes called a human interactive type electronic whiteboard, and is described in, for instance, Japanese Patent Application Laid-open Publication Kokai Hei 6-149531; Elrod et al, xe2x80x9cLiveboard: A large interactive display supporting group meetings, presentations and remote collaborationxe2x80x9d, CHI""92, Conference Proceedings, May 1992; and Perdson et al, xe2x80x9cAn electronic whiteboard for informal workgroup meetingsxe2x80x9d, INTERCHI""93, Conference Proceedings, April 1993, ACM, 391-398. In these known human interactive type dynamic display systems, the control of a displayed content is carried out by GUIs (graphical user interface) which are basically identical with the conventional GUIs used in a usual desktop type display system.
FIG. 1 shows an image displayed on the display panel of a known electronic whiteboard system, in which the displayed image on the whiteboard 101 is moved or scrolled up and down as well as right and left. On a display panel of the whiteboard 101, there are displayed tool bars 102 and 103 for effecting the up and down scroll as well as the right and left scroll. For instance, when a user wishes to scroll the displayed image upward, the user has to handle a pen-like member or electronic chalk 104 such that an upward scroll arrow 105a in the up and down scroll tool bar 102 displayed along a right-hand side of the whiteboard 101 is continuously pointed by the pen-like member. As long as the user points the up scroll arrow 105a, the displayed image is continuously scrolled upward. Similarly, when the use points at a downward scroll arrow 105b by the pen-like member 104, the displayed image is scrolled continuously in the downward direction.
Further, when rightward or leftward scroll arrows 106a or 106b in the tool bar 103 displayed along a bottom side of the whiteboard 101 is pointed at by the pen-like member 104, the image displayed on the display panel is scrolled in the right-hand or left-hand direction, respectively. In this scroll system, it is also possible to scroll the displayed image up and down or right and left by pointing a scroll button 107a or 107b in the tool bar 102 or 103 by means of the pen-like member 104 and dragging the pen-like member up and down or right and left. This type of GUI is substantially identical with that for the conventional desktop type personal computer.
FIG. 2 is a schematic view of an image displayed on the display panel 101 of the known electronic whiteboard system, in which the page up and page down functionality is effected. If the user wishes to effect the page up or page down, a page up region 108a or a page down region 108b is indicated by the pen-like member 104, said page up region and page down region being provided in a page up and down tool bar 108 which is displayed along a bottom side of the display panel 101.
In the manner explained above, in the known human interactive type electronic whiteboard, since the scroll bars 102 and 103 are provided at predetermined portions of the whiteboard 101, a part of the displayed image might be hidden by the user from people who are watching the whiteboard 101. When the electronic whiteboard with a large size is used in a relatively large lecture room of a university, some of students can not see the displayed image sufficiently such that students can not fully or clearly understand the contents of the displayed information. Further, when the scroll is performed by dragging the scroll button 107a or 107b, it is necessary to move the pen-like member 104 over a relatively long distance, i.e. from top to bottom or from left edge to right edge. For instance, when the displayed image is to be moved rightward from the condition illustrated in FIG. 1, it is necessary to drag the button 107b rightward up to the rightward arrow mark 106a. To this end, the user has to walk from the left hand end to the right hand end of the whiteboard. During this movement of the user, the displayed information is partly hidden by the user. This is also applied to the page up and down operation, because the page up region 108a and page down region 108b are provided at both ends of the bottom side of the whiteboard 101.
In the above mentioned Japanese Kokai Hei 6-149,531, there is described another known human interactive type electronic whiteboard system, in which the scroll may be carried out by dragging the pen-like member 104 on the display panel of the whiteboard 101. However, in such a system, when it is required to scroll the displayed image over a long distance, it is necessary to move the pen-like member over a very long distance on the display panel. It is apparent that such an operation requires a large movement of the user or user""s hand, and thus an amount of the displayed information hidden by the user becomes correspondingly large. Furthermore, on the electronic whiteboard, not only the text date stored in a memory is displayed, but also a graphical figure drawn by means of the pen-like member 104 is displayed. During this drawing operation, the scroll operation could not be performed, because the movement of the pen-like member 104 for the scroll could not be delineated from that for the drawing. Therefore, in such a case, before effecting the scroll, the draw mode has to be finished. Then, the operation of the user becomes very cumbersome.
In addition to the above scroll and page up and down functionalities, in the known electronic whiteboard system, there is also provided a functionality for incrementing or decrementing numerical data displayed in a counter 109 on the whiteboard 101 as illustrated in FIG. 3. On a right-hand side of the counter 109, there are displayed incrementing button 109a and decrementing button 109b. When the incrementing button 109a is pointed continuously or repeatedly by the pen-like member 104, the count value is increased one by one.
In the known electronic whiteboard system, since the count value incrementing or decrementing button 109a or 109b has to be continuously or repeatedly pointed by the pen-like member 104, a part of the displayed information might be continuously hidden by the user during this operation. Further, such an operation of the user is rather difficult.
Moreover, in the known electronic whiteboard system, a window displayed at a certain area on the display panel is moved or shifted into another area on the display panel. To this end, at first, an arrow mark displayed in a window shift tool bar is pointed by the pen-like member, and then the pen-like member is dragged on the display panel into a desired point. In this system, if the window is moved over a long distance, the user or user""s hand has to be moved over a long distance correspondingly. Therefore, an amount of the displayed information hidden by the user becomes correspondingly large and the operation of the user becomes cumbersome.
In order to mitigate the above mentioned drawbacks, it has been further proposed to use the so-called gesture interactive technique. The term xe2x80x9cgesturexe2x80x9d means a meta-stroke, i.e. a stroke that is not taken to be an element of the drawing, but is to be interpreted as a command. However, such a gesture technique has several problems as follows:
(1) Design of the gestures is very difficult and the gestures are often difficult to remember and easy to forget.
(2) The gestures have simple shapes, and thus could hardly be recognized by machines.
(3) There is little context to argument gestures recognition.
(4) Mode change for inputting the gestures is troublesome and easy to forget.
Due to the above problems, users fear that the gestures might be misinterpreted and may hesitate to use the gestures.
Therefore, the present invention has for its object to provide a novel and useful human interactive type display system, in which the above mentioned demerits of the known systems could be mitigated and desired functionalities can be performed with a minimum amount of displayed information hidden by a user, while a necessary movement of the user and user""s hand can be minimized.
According to the invention, a human interactive type display system comprises:
a display means having a display panel on which an image is displayed, said display panel having a predetermined operation region;
a marking means for pointing a point on said display panel; and
a controlling means for detecting a marking operation of a user by said marking means for a point within said operation region in the display panel to produce an enable signal and coordinates of points on the display panel pointed by said marking means successively from said point within the operation region to produce a position signal during a marking operation of the user, processing said enable signal and position signal to detect a direction and a distance of a movement of said marking operation of the user and controlling the image displayed on the display panel in accordance with said detected direction and distance of the movement of the marking operation to perform a desired functionality in a human interactive manner.
Although the present invention is not limited to an application to the human interactive type electronic whiteboard, such an application is particularly advantageous. In this human interactive type electronic whiteboard according to the invention, the display system comprises:
an electronic whiteboard having a display panel on which at least one operation region is set;
a pen-like member for indicating a point on said display panel;
a detection means for detecting a contact condition of the pen-like member to said operation region in the display panel and coordinates of a contact point to generate contact signal and coordinate signal; and
a control means for processing said contact signal and coordinate signal to producing an enable signal from said contact signal and a movement signal representing a direction and an amount of a movement of the pen-like member on the display panel from said coordinate signal, and changing an image displayed on said display panel in accordance with said enable signal and movement signal to perform a desired functionality in a human interactive manner.
In a first aspect of the invention, the scroll functionality is carried out by moving the image displayed on the display panel in a direction over a distance which is defined by the direction and amount of the movement represented by said movement signal.
In a preferable embodiment according to the first aspect of the invention, said detection means further detects a velocity of the movement of the pen-like member on the display panel, and said control means defines said distance over which the displayed image is to be moved by multiplying said distance represented by said movement signal by a coefficient which is determined in relation to said detected velocity of the movement. In this embodiment, the displayed image can be scrolled over a long distance by a small but fast displacement of the pen-like member on the display panel. Therefore, a portion of the displayed image hidden by the user can be made very small.
In a second aspect of the invention, the page up and down functionality is performed. In this case, the operation region is set to be identical with an area on which a text image such as a book is displayed. Further, only a horizontal or vertical component of the movement of the pen-like member is extracted and the page up or down is effected in dependence upon a sign or direction of the extracted horizontal component.
In a third aspect of the invention, a window displayed on the display panel is moved or shifted. In this case, a point within a predetermined region, e.g. a title bar area in the window is first pointed by the pen-like member, and then the pen-like member is moved or dragged on the display panel toward a desired area into which the window is to be shifted. Also in this window shift functionality, it is sufficient to drag the pen-like member over a small distance on the display panel, and therefore a portion of the displayed information hidden by the user during the window shift functionality can be made small.
According to a fourth aspect of the invention, a numerical value indicated by a counter displayed on the display panel can be incremented or decremented. In this case, a predetermined operation region near the counter is first pointed by the pen-like member, and then the pen-like member is dragged upward to increment the count value one by one or the pen-like member is dragged downward to decrement the count value one by one. Since the pen-like member is moved on the display panel during this count value changing functionality, the undesired affect of the user upon the displayed information is mitigated to some extent. Furthermore, the operation of the user becomes very easy as compared with the known count value changing operation. That is to say, in practice, it is difficult to tap or mark the same point repeatedly by the pen-like member.